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ABSTRACT 


Teaching office applications such as word processing, spreadsheet and presentation skills has been 
widely debated regarding its necessity, extent and delivery method. Training and Assessment 
applications such as MylTLab, SAM, etc. are popular tools for training students and are particularly 
useful in measuring Assurance of Learning (AOL) objectives. Meeting these assessment objectives has 
become a crucial issue in business schools as it now plays a major role in AACSB accreditation. It is 
our contention that these tools are fundamentally necessary to train and assess students to meet 
specific objectives that support a particular goal. In our experience, the simulation component of 
these tools is not enough to ensure all objectives. In this paper, we describe our experience with the 
use of in-the-application assignment projects to supplement the assessment and training simulation in 
order to improve final assessments and close the AOL loop. 
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Teaching office applications such as word 
processing, spreadsheet and presentation skills 
has been widely debated regarding necessity, 
extent and delivery method. Some contend 
that entering freshman should have had 
exposure to these applications and require the 
passing of an assessment exam (Shannon, 
2008). Others believe that high school exposure 


1. INTRODUCTION 


does not ensure necessary advanced skills in 
applications such as spreadsheets and require 
additional training (Hulick & Valentine, 2008). 
Traditional training in computer applications has 
generally included lecture and lab assignments 
in the particular application (Mykytyn, Pearson, 
Paul, & Mykytyn, 2008). In more recent years, 
many universities have turned to assessment 
and training tools such as MylTLab, SAM, 
SimNet and SNAP (Hill, 2011; Morris, 2010). 
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These tools require students to complete various 
tasks in a simulated application. The tools are 
also debated, as some wonder if students are 
really just learning to "click and point" to learn 
specific tasks but do not have the ability to 
actually apply these learned tasks to solve 
business problems (Coleman, Thrasher, & 
Atkinson, 2010). 

However, many universities not only use these 
simulation tools for training but also for 
implementing assurance of learning (AOL) 
standards mandated by the AACSB. Meeting 
these standards has become crucial, as they 
now play a major role in the AACSB 
accreditation of business schools (AACSB, 
2007). Program learning goals must be set, 
objectives must be measured across time, and 
the results used for continuous improvement 
(a.k.a "closing the loop" (Al-Mubaid, 
Abeysekera, Kim, Perkins-Hall, & Yue, 2011; 
Hollister & Koppel, 2006)). 

In this paper, we examine the extent to which 
these tools can be useful in attaining AOL 
objectives with regard to computer application 
skills. After providing a brief overview of the 
debate over teaching computer application skills, 
we look at how schools have responded with the 
use of automated training and assessment tools. 
We then relate our own school's experience with 
teaching computer applications, the use of these 
automated tools and how we supplemented their 
use in implementing the continuous 
improvement process necessary for our school's 
maintenance of AACSB accreditation. 

2. BACKGROUND 
Office Applications 

One assessment goal in many business schools 
is that students have the ability to use 
technology (Hollister & Koppel, 2007). Computer 
application skills in word processing, 
spreadsheets and presentations are vital for all 
business students as they matriculate and in 
future employment (Wolk, 2008). 

The need for business schools to teach these 
skills and/or assess a student's skill level has 
been a subject of discussion in many schools. 
One question usually discussed is "shouldn't 
incoming freshman have these skills?" The 
answer is that some do but many don't. 
Research indicates that a large percentage of 
students are not able to successfully pass a 
beginning assessment (Hulick & Valentine, 
2008; Shannon, 2008; Kline & Strickland, 
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2004), even in states where competency in 
technology is required for high school graduation 
(Grant, Malloy, & Murphy, 2009). This research 
also shows that students may overestimate their 
ability in office productivity tools. Students have 
a much higher perception of their level of skill in 
these applications than their actual performance 
on assessments (Grant, et al., 2009). Their 
study particularly indicated that students did not 
possess an adequate set of spreadsheet skills 
(as did (Kline & Strickland, 2004)). Thus, 
without curricular intervention of some sort, 
many students will not take a computer 
applications course and therefore continue to 
lack critical skills such as spreadsheets. 

Assessment Tools 

In order to ascertain that students obtain or 
have these computer application skills, 
universities have turned to training and 
assessment tools for test-out and instruction 
(Morris, 2010). Assessment and training tools 
have become quite popular in business programs 
to ensure that students have adequate skills in 
office production software, and to assess skill 
level and determine placement (Coleman, 
Thrasher, & Atkinson, 2010; Tesch, Murphy, & 
Crable, 2006). Currently, the most popular tools 
include MylTLab, SAM, SimNet and SNAP (Hill, 
2011 ). 

These tools offer many benefits: 

• Individualized instruction - students can 
work on modules that focus on skills in 
which they are deficient (Morris, 2010). 

• Consistent content across sections in 
multi-section courses - this also 
encourages consistency of results across 
sections (Kline & Strickland, 2004). 

• Automated grading is quick, and speeds 
the gathering of assessment data 
(Merhout, Benamati, Rajkumar, Anderson, 
& Marado, 2008). 

• Distance learning - automated tools can be 
incorporated in online courses relatively 
easily (Huan, Shehane, &Ali, 2011). 

However, as mentioned earlier, some question 
the effectiveness of these tools, and what few 
results have been reported have been mixed 
(Morris, 2010; Coleman, Thrasher, & Atkinson, 
2010; Paranto, Neumann, & Zhang, 2008; Kline 
& Strickland, 2004). 

Assurance of Learning 

The importance of assessment in business 
schools has increased significantly since 2003 
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when the AACSB adopted new standards for 
accreditation and reaccreditation. Prior to 2003, 
the AACSB had only 10% of the criteria related 
to assessment. Currently, one third of the 
standards are assessment-related (Pringle & 
Michel, 2007). 

Assessment has played such an important role in 
accreditation because stakeholders in 
universities such as state legislators, taxpayers, 
parents, donors and the federal government are 
requiring direct evidence of student learning 
(Bollag, 2006; Suskie, 2004). Computer 
application simulation tools can be used to easily 
measure relevant AOL objectives. The model in 
Figure 1 shows the loop that is referred to by 
the phrase "closing the loop", with regard to 
assurance of learning. Simulation tools can fill 
the assessment role depicted in Al-Mubaid, et 
al.'s (2011) model. See their paper for a 
complete description of the assessment process. 



Accreditation/course committee level 


Figure 1. Conceptual Model of Assessment 
(reprinted from Al-Mubaid, et al., 2011) 

3. ASSESSMENT STRATEGY 

Teaching Business Applications 

At our university, most faculty members in the 
school of business agree that a sound curriculum 
include a student's mastery of fundamental 
computer applications such as word, 
presentation, spreadsheet and database. 
Flowever, delivery of training for these tools 
continues to be widely debated. 

We informally surveyed nine of the largest 
schools by enrollment in the North Carolina state 


university system. Results show that schools 
address this delivery issue in a variety of ways: 

• Require all students to take a course. 

• Pass an assessment or take a course. 

• Incorporate computer application skills 
with a Management of Information 
Systems (MIS) course. 

• Pass an assessment initially or use a self- 
study application tool until passing the 
assessment. 

Additionally, the course and or courses have a 
variety of content including: 

• One course or assessment that includes 
word processing, spreadsheet and 
presentation applications (sometimes with 
an office database application such as 
Microsoft Access). 

• Separate courses for word 

processing/presentation and 

spreadsheet/database. 

• An MIS course that includes spreadsheets 
only. 

• An MIS course that includes spreadsheets 
and database applications. 

Although this data is limited in scope and size, it 
can reasonably be assumed that other business 
schools debate the best way to ascertain the 
delivery of application skills. Over the past 
several years our university has used a variety 
of delivery modes. In Fall 2008 and Spring 2009, 
we offered one business computer applications 
course that included word processing, 
spreadsheets and presentation skills and a 
separate course that includes office database 
applications. 

At this writing, our first business computer 
applications course includes only spreadsheet 
skills. The decision to not teach word processing 
and presentation is largely based on student's 
requirement to have these skills in other courses 
and their ability to learn these skills on their 
own. Additionally, incoming freshman do not 
have the ability to complete even basic 
spreadsheet tasks (Grant, et al., 2009; Kline & 
Strickland, 2004) and these skills are deemed 
vital for matriculation and post-graduation 
employment. 

Students can test out of the first business 
computer applications course (Microsoft Excel). 
The database application course is an elective. 
Our teaching and assessment tool is Pearson's 
MylTLab. All business students are required to 
obtain a score of 70% or better on an 
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assessment or take the business applications 
course. This percentage is based on the 
business school policy of requiring students to 
matriculate with a C or better in all their 
coursework. All students who take the course 
are required to take the same assessment as a 
post-test. 

In the Fall of 2008 and Spring of 2009 this 
assessment included the testing of 10 MS-Excel 
skills, 5 MS-Word skills and 5 MS-PowerPoint 
skills. In the Fall of 2009 the pre- and post-test 
assessed 20 MS-Excel skills. Seven MS-Excel 
skills are persistent during the entire test period 
of Fall 2008 to Spring 2011. Each of these tested 
skills includes 2-5 tasks. All of the tasks for 
each skill must be completed successfully. 

AOL Goal 

We use this pre- and post-test of all business 
students to measure our technology AOL goal. 
The criterion for meeting this goal is that 70% of 
the students correctly complete each skill tested. 
A summary of our AOL report is included in 
Appendix A. 

Each semester's post-tests are reviewed and a 
strategy to reach our goal of 70% on all skills is 
determined. In spite of several strategies, our 
post-test results in Fall 2008 through Spring 
2009 indicated that, on average, half of the 
original 10 tested skills were below standard. 

Project Implementation 

A criticism of the assessment and application 
tools is that students only learn to click and 
point in a simulated environment and these skills 
do not always translate to "in-the-application" 
skills. Project-based courses in business 
applications may be more successful (Murray, 
Hooper, & Perez, 2007) but are not always 
practical in terms of training large numbers of 
students. 

After the initial introduction of the MylTLab tool, 
Pearson Education received numerous requests 
for a built-in grader for problem solving projects 
that could be performed in the actual 
application. In Fall 2009, MylTLab offered an 
applications enhancement called Project Grader. 
This enhancement offered in-the-application 
projects. Students would download a beginning 
spreadsheet and perform a variety of tasks in 
MS-Excel, upload the completed spreadsheet 
and receive a grade based on the correct 
completion of those tasks. 
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In order to determine what effect the projects 
would have on the overall performance of 
students on the final assessment, we 
implemented projects in one section over two 
consecutive semesters (Fall 2009, Spring 2010). 
Projects were implemented in all sections in the 
Fall 2010 semester. Instructors determined how 
many projects to include in their section. In 
Spring 2011, all sections included 7 projects in 
addition to the simulation training. See Appendix 
B for the results of these sections. 

4. WHAT WE LEARNED 

Teaching and/or assessing students in computer 
applications skills and measuring our AOL 
objectives remain an ongoing process. However, 
our experience has taught us that: 

• Incoming freshmen do not always have 
the necessary computer application skills, 
in particular spreadsheet skills. 

• Training and Assessment simulation tools 
have proven to be an effective method for 
training students and measuring AOL 
objectives. 

• Augmenting simulation training with 
projects that require the use of the actual 
spreadsheet application improves AOL 
measured objectives. 

The average compliance improved each 
semester except one. However the most 
dramatic increases in the percentage of correct 
tasks were in the one section using 7 projects in 
Fall 2009 (see Table 2). When 3 projects were 
used in one section, improvement was noted in 
some skills but not in others. When instructors 
determined how many projects to implement in 
Fall 2010, results were mixed. In Spring 2011, 
all sections implemented the 7 projects used in 
Fall 2009 and all persistent skills (of the original 
10) tested met the standard. Although our data 
is not scientific proof that adding projects, in 
particularly these 7 projects, increases a 
student's overall skill level, it gives us a base for 
improvement. Additionally, we show continued 
improvement over time. 

Simulation tools are extremely useful especially 
in assessing and training computer applications 
to large numbers of students. Additionally, 
students' acceptance of this type of training is 
high (Baker, 2004). However, simulation training 
may not completely prepare students to 
successfully apply the skills learned to later 
tasks and projects using computer application 
skills. Project-based training in-the-application 
only is not practical in terms of time and 
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resources for large numbers of students. In our 
experience, a combination of simulation training 
and in-the-application training increases the 
likelihood that students will be able to complete 
any given task in that application. 

5. MOVING FORWARD 

Our experience supports previous research that 
project-based courses in computer application 
increases the skill level of the students. 
Specifically, the addition of application-based 
projects in our courses increased the percentage 
of students who could successfully complete the 
tasks tested and closed the loop for our 
technology AOL goal. Meeting AOL goals for 
AACSB accreditation is vital for business schools. 
Evaluating our assessment goals every semester 
and supplementing simulation training with live 
application projects significantly increased our 
ability to "close the loop." 

We plan to continue using these projects and 
measure the student's success with the 
additional 13 skills in the pre- and post-tests 
(please contact author for a list of these skills). 
Additionally, based on the pre- and post-tests, 
we will adjust the project focus to tasks that 
specifically address the desired skill. 
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